Expand description

Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

Usage (with the RPhi implementation crate)

use rphi::prelude::*;

#[tokio::main]
async fn main() {
    let mut model = Phi::default();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Re-exports

Structs

Enums

Traits

  • A model that has a chat format.
  • A model that can be created asynchronously.
  • A model that can be used to embed text. This trait is generic over the vector space that the model uses to help keep track of what embeddings came from which model.
  • A model that can be used to generate text with an associated tokenizer.
  • An extension trait for models.
  • A session for a model.
  • An extension trait for Streams that provides a variety of convenient combinator functions.
  • A raw interface for a model that can be used to generate text synchronously. This provides a very low level interface to a model’s session:
  • An extension trait for sync models.
  • The type of a vector space marks what model the vector space is from. You should only combine vector spaces that come from the same model.

Type Aliases